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Abstract. In this paper, we propose a computational framework for 3D volume 
reconstruction from 2D histological slices using registration algorithms in feature 
space. To improve the quality of reconstructed 3D volume, first, intensity varia- 
tions in images are corrected by an intensity standardization process which maps 
image intensity scale to a standard scale where similar intensities correspond to 
similar tissues. Second, a subvolume approach is proposed for 3D reconstruction 
by dividing standardized slices into groups. Third, in order to improve the quality 
of the reconstruction process, an automatic best reference slice selection algorithm 
is developed based on an iterative assessment of image entropy and mean square 
error of the registration process. Finally, we demonstrate that the choice of the 
reference slice has a significant impact on registration quality and subsequent 3D 
reconstruction. 



1. Introduction 

2D imaging methods, such as optical microscopy, are still preferable to 3D imag- 
ing methods due to their high level of specificity and high resolution properties. 
Histological sections (slices) obtained through 2D imaging methods provide use- 
ful information for the diagnosis or the study of pathology. Although 2D histologi- 
cal slices have great impacts on quantification and visualization of clinical data, 3D 
volume reconstruction from these 2D slices is required in order to fully appreciate 
anatomical structures JT). 

Typically a 3D volume is reconstructed by registering (aligning) the 2D sections 
with respect to a chosen reference and stacking successive aligned sections 0. 
As the acquisition processes of different 2D histological images are performed in- 
dependently slice misalignment and deformation is often unavoidable. The de- 
formation varies from section to section and non-cohorent distortions may exist 
in consecutive sections. Choosing an arbitrary slice as a reference slice leads to 
errors in 3D volume reconstruction, hence, the reference slice should be chosen 
properly not to contain distortions in order to achive high quality volume recon- 
struction OH). 

Fully automatic registration of histological slices and reconstruction of 3D vol- 
ume are necessary for two reasons. First, since manual registration using interac- 
tive alignment is non-reproducible and user dependent, it cannot be used if the 
number of slices is large [l}|2). To quantify changes between images, motion and 
deformation characteristics specify the type of transformation (registration). Since 
histological slices change smoothly from slice to slice and the section distortions 
induced by the preparation process are local in nature GUSHD, accurate alignment 
of these slices can be achived by using elastic registration methods. Second, since 
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manual selection of the best reference slice (BRS) uses qualitative measures and ig- 
nores the image information content, optimum smooth 3D volume reconstruction 
cannot be guaranteed. 

In this paper, we present a fully automatic 3D reconstruction method which 
tackles three difficult problems in registration of histological images. Section|2]ex- 
plains an important preprocessing method, called standardization, which captures 
intensity variations between slices and plays a significant role in identifying BRS 
selection. In Section |3j the edgeness space is presented for the registration frame- 
work to provide better global alignment and to avoid possible misalignments. Sec- 
tion S] briefly explains locally affine globally smooth (LAGS) registration method. 
Based on iterative assessment of image entropy and Mean Square Error (MSE) of 
the registration process in feature space, an automatic BRS selection algorithm is 
described in Section [5] To evaluate reconstructed volume qualitatively and quan- 
titatively, we use both Correlation Alignment Measure (CAM) and proposed Stan- 
dard Deviation Maps (SDM) in Section [6] Evaluations and discussions are given 
in Section[7l 



2. Standardization of Image Intensity Scale 

Image intensity variations are not only influenced by the distribution of light 
sources, but also the content (different tissues) of the images as different tissues 
show different intensity levels. To avoid the effects of illumination conditions and 
identify those intensity variations due to different tissue types, a standardization 
procedure is applied to histological images. 

Standardization is a non-linear pre-processing technique which maps image in- 
tensity histogram (scale) into a standard intensity histogram (scale) so that similar 
intensities will have similar tissue meaning after standardization. Standardization 
was firstly developed for MR images [9J. In previous works OH), we applied this 
method to standardize histological images. 



3. Feature Space 

Choice of the feature space plays a significant role in image registration espe- 
cially if the similarity metric is based on the optimization function independent of 
spatial information such as mutual information. Since these kind of registration 
methods do not take into consideration the spatial information of pixel/ voxel in- 
tensity distribution/ variation, the optimization algorithm may get stuck in local 
maximum resulting in misalignment. Defining a feature space capturing varia- 
tions of gray-level characteristics will overcome the drawbacks of intensity based 
approaches. To align the images globally, we used a particular feature space which 
represents an image by continuous variables, called edgeness, and describes the 
intensity variance of a predefined region over the image |10, 3, 4J. 

We represent an image (section/ slice/scene) by a pair F = (F, g) where F is a 
two-dimensional (2-D) array of scene elements (pixels) and g is intensity function, 
whose domain is F. We assign an integer intensity value for each pixel o G F. 
Edgeness feature space is defined by the pair F e = (F, r/) = (F, g,rf), where r/ is 
a fixed radius for each region. At image coordinate ?*o, the edgeness is represented 
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by 

(1) F e = £ \g(n) - g(r )l 

\ r i~ r o\< r f 

where r/ is the radius. It should not be concluded that the process is just empha- 
sizing edges and deciding whether a specific voxel/pixel belongs to the edge or 
not fl0j|3j|4]|. Instead, within a specified radius value, the image feature content 
is forced to stay beyond a variation level which prevents the registration process 
from getting stuck in local maxima. 

4. Locally Affine Nonlinear Transformation 

Local alignment (elastic) of images is obtained through Locally Affine Globall 
Smooth (LAGS) registration method described in [7, 8, 3 1. Since consecutive slices 
are not exactly the same, rather slices vary smoothly, LAGS registration algorithm 
fits well to the problem. For 2D images, 8 affine parameters are needed to fully 
identify changes between images. Two of these affine parameters are needed to 
capture local brightness and contrast patterns [7J and 6 affine parameters are used 
to capture local deformations for 2D images. Since the standardization procedure 
has been used to remove intensity variations among the same tissue types, there 
is no need to use these 2 affine parameters. Briefly, LAGS registration algorithm 
uses the difference image of source and target image as an optimization function 
and tries to minimize it over small local image domain. Readers are strongly en- 
couraged to read [7, 8, 3, 4J to understand the theory of LAGS and the modified 
algorithm which takes into account the standardization procedure. 

5. Automatic Best Reference Slice Selection 

The quality of the 3D volume reconstruction process mostly depends on the 
choice of the reference slice. The reference slice is used as a target image and 
all the remaining slices are being considered as source images to be registered 
onto the target image. If the reference slice is distorted or noisy, reconstructed 3D 
volume will not be optimal. Once the reference slice is identified as target image, 
registration based fusion methodology can be applied for reconstruction [2J. 

Selecting best reference slice can be based on high confidence image features 
such as MSE, entropy, edge, texture, color, intensity histograms, etc. 

(1) MSE: In the case of distortions, structural discontinuity is not minimum 
even for the consecutive slices. When affine registration is performed for 
global alignment of images, the optimization procedure tries to minimize 
MSE between images but due to distortion, it will not reach low MSE val- 
ues. Furthermore, it is also known that with small SNR values, alignment 
is difficult, leading to high registration errors. Therefore, MSE can be used 
as a tool for checking whether the slices are distorted or not. While high 
MSE values indicate most probably distorted and noisy slices, low MSE 
values indicate strong similarity between consecutive images. 

(2) Edge: In feature space, we emphasise edgeness features of an image by 
mapping image space into the feature space where edgeness parameters 
hold both edge information and spatial variations of pixel intensities over 
all regions in the image. Therefore, we assume that MSE between any 
image pair already includes high confidence information related to edges. 
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(3) Contrast/Brightness: Contrast/brightness patterns also play an important 
role in image contents. Since the standardization method has been used to 
correct intensity variations, intensity for the same tissue is the same for all 
images. 

(4) Entropy: Entropy is another measure often used to characterise the infor- 
mation content of a data source. It has been used as a metric for image 
registration in the form of mutual information. Large mutual information 
between images implies high similarity and vice versa. 

To select BRS automatically, one needs to define a metric which describes the 
reference slice in terms of noise, distortion and information content levels. As 
m EBj, BRS can be formulated as 

where MSEij is the feature space based mean square error after registering the 
image i into the image j and E is entropy. Further theoretical details of BRS selec- 
tion algorithm can be found in 01!]. 

6. Implementation, Experiments and Results 

Registration of histological slices requires serial registration procedure which 
is just a combination of transformation functions. Let {A,^|z < j} be the trans- 
formation function that warps the source image i into the target image j. The 
transformation is computed serially as follows 

Aj^i = Aj < _j_i o o ... A i+1 ^i, for i < j 

(3) Aj^i = Aj^ j+1 o Aj + i<_j +2 o . . . Ai-ii-i, for i > j, 

where o represents the composition. 

One of the advantages of using the subvolume approach is to avoid the "banana 
shape" effect resulting from the reconstuction process (TJ[2). In serial registration, 
one of the disadvantages of using affine registration is that we lose the topology 
of the reconstructed volume. To reduce or eliminate this effect, one may need to 
use MRI of the volume superimposed onto the reconstructed histological volume 
or use rigid registration. 

In summary, registration is performed initially for slices in each subvolume sep- 
arately. Three kinds of registration are performed in the reconstruction process: 
rigid, affine and LAGS. MSEs are calculated according to affine registration in ed- 
geness space and have been used to select BRSs for each subvolume. Affine regis- 
tration is performed in a serial manner combining transformation functions. Then, 
LAGS registration is performed to capture local deformations in each subvolume 
with respect to the chosen reference. Once LAGS registration has been finished, 
subvolumes are registered to each other in a rigid manner. 

6.1. Evaluations and Results. We have registered a stack of 350 Nissl-stained 
slices acquired by cyro-sectioning coronally on an adult mouse brain with a reso- 
lution of 590x520 pixels at a resolution of 15/im and 24-bit color format 1 11 1. 

Quantitative evaluation of the results of the reconstruction process is often dif- 
ficult. It has been shown in [12 J that an ideal measure of the quality of the re- 
construction is the smoothness of the reconstructed surfaces. In this work, they 
propose a new measure based on evaluation of smoothness of the reconstructed 
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volume called Correspondence Alignment Measure (CAM). As an alternative method 
to CAM, Standard Deviation Maps (SDM) is proposed to measure the smoothness of 
the reconstructed volume. The method is based on the standard deviation of the 
pixel values for the same location in each section. The CAM and SDM results for 
the reconstructed 3D volume and validation of the SDM are given in the following 
subsections. 

6.1.1. CAM. The CAM measure relies on the assumption that if a point is perfectly 
aligned, it lies midway between its corresponding points on neighbors' sections. 
To compute the CAM measure for a given image, first of all, corresponding points 
for specifed control points in the image are identified. The associated confidence 
values in two adjacent images are then calculated. If the confidence is greater than 
a pre-defined threshold r, square root of the summation of the deformation vectors 
are added to the cumulative sum. Finally, the cumulative sum is normalized by 
the number of pixels which have contributed. Note that CAM gives one value 
for each image, therefore, mean or standard deviation of CAM values of serial 
images are needed to compare reconstructions. Reconstucted volume is smooth if 
the mean or the standard deviation of CAM measures are low and vice versa. 

Summary of the changes in mean and standard deviation in CAM values is 
given by Table [T] The values in Table Q] are obtained by considering the worst case 
which uses all the slices instead of just a few slices from the middle of the stack 
as defined in ||T2|. and r is set to 0. Even for the worst case, CAM values indicate 
that a smooth volume is constructed with the proposed framework. While mean 
values dropped by 7.29% and 18.69%, the standard deviation values dropped by 
24.46% and 27.73% for affine and locally affine registered stacks respectively. 
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FIGURE 1 . CAM for original and registered slices 



6.1.2. SDM. We offer here a simple way of measuring the quality of the recon- 
struction by considering the smoothness of the reconstructed surfaces. If the recon- 
structed volume is naturally smooth, it means that the structures change smoothly 
and slowly from slice to slice which highly depends on the registration quality. 
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TABLE 1. CAM-mean and standard deviation values for recon- 
structed 3D 





Rigid Reg. 


Affine Reg. 


LAGS Reg. 


Mean 


55.911 


51.832 


45.461 


Std 


12.223 


9.232 


8.833 



Assumption: For any reconstucted volume V, which is a sequence of images 
j — 1, .., M, if we take the same pixel position v for all slices in F, one should 
expect smooth transition of pixel values within the slices of F, if the slices are well 
registered. 

To validate this assumption experimentally, a 3D MRI volume MV is taken 
and its slices are warped by applying random deformations spanning high-to- 
low level. The randomly warped slices are used to reconstruct a warped 3D MRI 
volume MV W for which we compute SDM and compare it with the smoothness of 
MV. 

Figure |2] shows the experimental validation of SDM for different level of defor- 
mations applied to the slices of MV. While the first SDM (the first image in the 
first row) has carried the highest deformation level, the level of deformation has 
been decreased until the last SDM (the third image in the second row) is obtained, 
which carries no deformation level, MV itself. As the deformation level decreases, 
the smoothness level of MV W approaches the original volume MV. SDMs for the 




FIGURE 2 . SDMs for warped volumes spanning from high-to-low 
level of deformations 

reconstructed volume MV W for w =rigid, affine and LAGS are shown in Figured 
both in gray scale and spectrum format. Although the volume reconstructed by 
successive affine registrations is smoother than the volume reconstructed by suc- 
cessive rigid registrations, it includes the "banana-shape" effect which can be cor- 
rected by superimposing MRI of the rat brain (if available) on the reconstructed 
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FIGURE 3. Standard Deviation Maps for Rigid registered, Affine- 
Registered and Elastic-Registered Stacks 



7. Conclusion 

In this paper, we have presented a novel framework to reconstruct 3D rat brain 
volume from 2D histological images. The framework is based on three funda- 
mental premises. (1) All histological images must be standardized for accurate 
registration leading to 3D volume reconstruction. (2) For accurate and succesful 
registrations in consecutive slices, a reliable feature space must be taken into ac- 
count. (3) For automatic 3D volume reconstruction, the reference slice must be 
chosen properly by avoiding slices with high noise, distortions and other factors. 
To validate the reconstructed volume, the smoothness of the volume is considered. 
In addition to the existing method CAM, we have proposed a method called SDM 
to measure the smoothness of the reconstructed volume. Qualitative and quanti- 
tave evaluation of experimental results indicate that the reconstructed volume is 
highly accurate. 
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